feature: Adding Training Compiler support for TensorFlow estimator starting TF 2.9 #3156

Lokiiiiii · 2022-06-02T23:06:08Z

Issue #, if available:

Description of changes:
As title
This is a fast follow up to #3154

Testing done:
Local testing and PR testing

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2022-06-02T23:09:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 05d0895
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-02T23:24:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ef919b3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-02T23:30:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d676d3a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-02T23:48:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: befb150
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-06-02T23:48:23Z

Codecov Report

Merging #3156 (56ca880) into master (325214f) will decrease coverage by 0.05%.
The diff coverage is 93.22%.

@@            Coverage Diff             @@
##           master    #3156      +/-   ##
==========================================
- Coverage   89.70%   89.64%   -0.06%     
==========================================
  Files        1048      202     -846     
  Lines       87841    17442   -70399     
==========================================
- Hits        78794    15636   -63158     
+ Misses       9047     1806    -7241

Impacted Files	Coverage Δ
src/sagemaker/training_compiler/config.py	`84.21% <77.77%> (-1.91%)`	⬇️
src/sagemaker/tensorflow/estimator.py	`92.80% <81.81%> (-1.06%)`	⬇️
src/sagemaker/huggingface/estimator.py	`93.02% <100.00%> (ø)`
.../sagemaker/huggingface/training_compiler/config.py	`100.00% <100.00%> (ø)`
src/sagemaker/image_uris.py	`98.49% <100.00%> (+0.48%)`	⬆️
src/sagemaker/tensorflow/__init__.py	`100.00% <100.00%> (ø)`
...c/sagemaker/tensorflow/training_compiler/config.py	`100.00% <100.00%> (ø)`
...ackages/sagemaker/cli/compatibility/v2/__init__.py
...ages/sagemaker/feature_store/feature_definition.py
... and 846 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 325214f...56ca880. Read the comment docs.

sagemaker-bot · 2022-06-02T23:54:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 05d0895
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:09:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: befb150
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:11:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ef919b3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:14:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: d676d3a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:17:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ef919b3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:17:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 05d0895
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:20:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 05d0895
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:21:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: befb150
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:21:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d676d3a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:23:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: befb150
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:28:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ef919b3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:32:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: d676d3a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:41:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: befb150
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:47:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: cbac484
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T00:50:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 320d05b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T01:00:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 320d05b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T01:11:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: cbac484
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T01:14:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 320d05b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T01:20:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 320d05b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T01:22:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: cbac484
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T01:29:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: cbac484
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T01:40:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 320d05b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T06:00:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 9accb69
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-03T06:09:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 9accb69
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T03:57:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 84a6b00
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T04:16:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 84a6b00
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T05:43:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 80a223a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T06:14:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 80a223a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T06:26:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: df4d6d9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T06:26:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 80a223a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T06:28:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 80a223a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T06:44:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 80a223a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T06:46:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: df4d6d9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T06:55:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: df4d6d9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T07:09:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: df4d6d9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T07:29:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: df4d6d9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T17:31:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 56ca880
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T17:43:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 56ca880
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T17:54:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 56ca880
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T18:16:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 56ca880
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-15T18:21:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 56ca880
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/training_compiler/config.py

src/sagemaker/image_uris.py

…arting TF 2.9 (aws#3156) Co-authored-by: Miyoung <[email protected]>

Lokiiiiii force-pushed the tf-2.9-trcomp branch from ef919b3 to d676d3a Compare June 2, 2022 23:26

Lokiiiiii added 3 commits June 14, 2022 22:38

Adding logic to convert compiler_config to hyperparameters

9f4cf52

Fixing trcomp tensorflow tests

a4d86c1

Fixing logic error in training_compiler supported version

80a223a

Fixing logic error in training_compiler

df4d6d9

Fixing logic error in training_compiler

56ca880

Lokiiiiii marked this pull request as ready for review June 15, 2022 18:47

navinsoni reviewed Jun 15, 2022

View reviewed changes

src/sagemaker/training_compiler/config.py Show resolved Hide resolved

src/sagemaker/image_uris.py Show resolved Hide resolved

Lokiiiiii changed the title ~~Adding Training Compiler support for TensorFlow estimator starting TF 2.9~~ feature: Adding Training Compiler support for TensorFlow estimator starting TF 2.9 Jun 15, 2022

navinsoni approved these changes Jun 15, 2022

View reviewed changes

shreyapandit approved these changes Jun 15, 2022

View reviewed changes

navinsoni merged commit d9463d3 into aws:master Jun 15, 2022

plumdog mentioned this pull request Aug 2, 2022

fix: add missing Tensorflow 2.9 inference image #3251

Closed

9 tasks

jerrypeng7773 pushed a commit to jerrypeng7773/sagemaker-python-sdk that referenced this pull request Aug 8, 2022

feature: Adding Training Compiler support for TensorFlow estimator st…

47e1453

…arting TF 2.9 (aws#3156) Co-authored-by: Miyoung <[email protected]>

feature: Adding Training Compiler support for TensorFlow estimator starting TF 2.9 #3156

feature: Adding Training Compiler support for TensorFlow estimator starting TF 2.9 #3156

Conversation

Lokiiiiii commented Jun 2, 2022 • edited Loading

Merge Checklist

General

Tests

sagemaker-bot commented Jun 2, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 2, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 2, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 2, 2022

AWS CodeBuild CI Report

codecov-commenter commented Jun 2, 2022 • edited Loading

Codecov Report

sagemaker-bot commented Jun 2, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 3, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 15, 2022

Lokiiiiii commented Jun 2, 2022 •

edited

Loading

codecov-commenter commented Jun 2, 2022 •

edited

Loading